{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Uploading Factor Risk Models\n",
    "\n",
    "The GS Quant `FactorRiskModel` class gives users the power to upload their own risk models to Marquee for seamless integration with the Marquee Portfolio Analytics and Plot Tool Pro suite. After uploading a custom `FactorRiskModel`, users can access their factor model data programmatically using GS Quant, visualize their factor risk model data with Plot Tool Pro, or run historical factor attribution analysis on equity portfolios through the lens of their uploaded factor risk model with GS Quant's `Portfolio` class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Authenticate and Initialize Your Session\n",
    "\n",
    "First you will import the necessary modules and add your client id and client secret."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "\n",
    "from gs_quant.session import GsSession, Environment\n",
    "from gs_quant.models.risk_model import (\n",
    "    FactorRiskModel,\n",
    "    RiskModelCalendar,\n",
    "    Term,\n",
    "    CoverageType,\n",
    "    UniverseIdentifier,\n",
    "    FactorType,\n",
    "    RiskModelFactor,\n",
    ")\n",
    "import datetime as dt\n",
    "\n",
    "\n",
    "client = None\n",
    "secret = None\n",
    "\n",
    "## External users must fill in their client ID and secret below and comment out the lines below\n",
    "\n",
    "# client = 'ENTER CLIENT ID'\n",
    "# secret = 'ENTER CLIENT SECRET'\n",
    "\n",
    "GsSession.use(Environment.PROD, client_id=client, client_secret=secret)\n",
    "warnings.filterwarnings(\"ignore\", category=RuntimeWarning)\n",
    "\n",
    "print('GS Session initialized.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Create a Factor Model\n",
    "\n",
    "Input fields to create the initial Factor Risk Model object\n",
    "\n",
    "| Attribute                        | Can be Modified? | Description                                                                                                                                                                                                                                                       \n",
    "|----------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
    "| id                               | No               | Model ID                                                                                                                                                                                                                                                          |\n",
    "| name                             | Yes              | Name of model                                                                                                                                                                                                                                                     |\n",
    "| description                      | Yes              | Longer description of model                                                                                                                                                                                                                                       |\n",
    "| term                             | Yes              | Term or horizon of model. One of: Long, Medium, Short, Trading, Daily                                                                                                                                                                                             |\n",
    "| coverage                         | Yes              | Geographical coverage of assets within model universe. One of: Global, Region, Region Excluding Countries, Country                                                                                                                                                |\n",
    "| vendor                           | Yes              | Who creates the model                                                                                                                                                                                                                                             |\n",
    "| version                          | Yes              | Version of model                                                                                                                                                                                                                                                  |\n",
    "| universe_identifier              | No               | Identifier used to upload the model's asset universe. One of: sedol, cusip, isin, bcid, gsid                                                                                                                                                                      |\n",
    "| expected_update_time  (optional) | Yes              | Time (in UTC) data is expected to be published.                                                                                                                                                                                                                   |\n",
    "| entitlements (optional)          | Yes              | Who can view, execute (display access in Marquee), query (via API), or upload data to the risk model. By default, the application that creates the risk model is granted admin entitlements, which include all these rights and the ability to edit entitlements. |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model_id = 'VENDOR_COVERAGE_MODELNAME_TERM_VERSION'\n",
    "model_name = 'My Risk Model'\n",
    "description = 'My Custom Factor Risk Model'\n",
    "term = Term.Medium\n",
    "coverage = CoverageType.Country\n",
    "universe_identifier = UniverseIdentifier.sedol\n",
    "expected_update_time = dt.time(6, 0, 0)\n",
    "vendor = 'Goldman Sachs'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the Factor Risk Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = FactorRiskModel(\n",
    "    id_=model_id,\n",
    "    name=model_name,\n",
    "    description=description,\n",
    "    coverage=coverage,\n",
    "    term=term,\n",
    "    universe_identifier=universe_identifier,\n",
    "    vendor=vendor,\n",
    "    expected_update_time=expected_update_time,\n",
    "    version=1,\n",
    ")\n",
    "\n",
    "model.save()\n",
    "print(f\"Successfully create a factor risk model with ID: {model.id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A note on Model IDs\n",
    "\n",
    "Model IDs are unique identifiers in Marquee that adhere to a standardized format, ensuring clarity and consistency for clients. The format is:\n",
    "\n",
    "`VENDOR_COVERAGE_MODELNAME_TERM_VERSION`\n",
    "\n",
    "- **Vendor**: The name of the entity or organization creating the model.\n",
    "- **Coverage**: The geographical asset coverage of the model (e.g., \"GLOBAL\", \"APAC\", \"EU\", \"US\", \"UK\", \"EM\" (or Emerging), \"DM\" (for Developed Markets), \"WW\" (for World Wide) etc).\n",
    "- **Term**: 1-3 character(s) denoting the time horizon of the model (e.g., \"L\", \"S\", \"M\", \"TRD\" for \"Long\", \"Short\", \"Medium\", or \"Trading\" respectively). Variations of these can also take the form of \"MH\" or \"MT\" for Medium Horizon and Medium term. \n",
    "- **Version**: The version number of the model, starting from 1 and incrementing with updates.\n",
    "\n",
    "This format is designed to be intuitive and easily recognizable for clients, making it simple to identify the purpose and scope of a model at a glance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Upload a Calendar To Your Model\n",
    "\n",
    "The calendar associated with the Factor Risk Model contains the dates which the risk model should have posted data on to be considered \"complete.\" The calendar can go further back as well as forward in time than the data that is currently posted for the calendar, but there cannot be any gaps in the data posted to the risk model according to the calendar. Please note that the `upload_calendar` function overwrites the previous calendar if one exists, so be sure to include all dates you want in the calendar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "calendar = RiskModelCalendar(\n",
    "    [\n",
    "        '2021-01-29',\n",
    "        '2021-01-28',\n",
    "        '2021-01-27',\n",
    "        '2021-01-26',\n",
    "        '2021-01-25',\n",
    "        '2021-01-22',\n",
    "        '2021-01-21',\n",
    "        '2021-01-20',\n",
    "        '2021-01-19',\n",
    "        '2021-01-18',\n",
    "        '2021-01-15',\n",
    "        '2021-01-14',\n",
    "        '2021-01-13',\n",
    "        '2021-01-12',\n",
    "        '2021-01-11',\n",
    "        '2021-01-08',\n",
    "        '2021-01-07',\n",
    "        '2021-01-06',\n",
    "        '2021-01-05',\n",
    "        '2021-01-04',\n",
    "        '2021-01-01',\n",
    "    ]\n",
    ")\n",
    "\n",
    "model.upload_calendar(calendar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to update a calendar? \n",
    "To update a calendar, you can use the same `upload_calendar` method with a new list of dates. The new list will replace the existing calendar. If you want to add new dates without removing the existing ones, you must first retrieve the current calendar using `model.get_calendar()` and then append the new dates before uploading.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the existing calendar\n",
    "existing_calendar = model.get_calendar().business_dates\n",
    "\n",
    "# Append new dates\n",
    "new_dates = ['2021-01-30', '2021-01-31']\n",
    "updated_business_dates = list(existing_calendar) + new_dates\n",
    "\n",
    "model.upload_calendar(RiskModelCalendar(updated_business_dates))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Upload Data To Your Model\n",
    "\n",
    "Once the calendar is posted for a model, we can start uploading data to it.\n",
    "\n",
    "The data must be uploaded in a specific format, which is defined in the `RiskModelData` class. The data must be uploaded for each date in the calendar. \n",
    "\n",
    "Below are the components of RiskModelData\n",
    "\n",
    "#### 1. Factor Data\n",
    "\n",
    "\n",
    "| Field            | Description                                                                                                                                             \n",
    "|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------\n",
    "| factorId         | Unique ID of the factor (must be < 20 characters). It must map consistently to the same factor across every date.                                       |\n",
    "| factorName       | Name of the factor. It can be any string and should be consistent across every date.                                                                    |\n",
    "| factorCategoryId | Unique Id of the category that the factor belongs to (must be < 20 characters). It must map consistently to the same factor category across every date. |\n",
    "| factorCategory   | Name of the category that the factor belongs to (Style, Industry, Market, Currency, etc.).                                                              |\n",
    "| factorReturn     | Daily return of the factor in percent units (i.e. for 10%, use 10).                                                                                     |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_data = [\n",
    "    {\n",
    "        \"factorId\": \"1\",\n",
    "        \"factorName\": \"Factor 1\",\n",
    "        \"factorCategoryId\": \"RI\",\n",
    "        \"factorCategory\": \"Style\",\n",
    "        \"factorReturn\": 0.39,\n",
    "    },\n",
    "    {\n",
    "        \"factorId\": \"2\",\n",
    "        \"factorName\": \"Factor 2\",\n",
    "        \"factorCategoryId\": \"RI\",\n",
    "        \"factorCategory\": \"Style\",\n",
    "        \"factorReturn\": 1.99,\n",
    "    },\n",
    "    {\n",
    "        \"factorId\": \"3\",\n",
    "        \"factorName\": \"Factor 3\",\n",
    "        \"factorCategoryId\": \"RI\",\n",
    "        \"factorCategory\": \"Style\",\n",
    "        \"factorReturn\": 0.29,\n",
    "    },\n",
    "    {\n",
    "        \"factorId\": \"4\",\n",
    "        \"factorName\": \"Factor 4\",\n",
    "        \"factorCategoryId\": \"MKT\",\n",
    "        \"factorCategory\": \"Market\",\n",
    "        \"factorReturn\": -0.9,\n",
    "    },\n",
    "    {\n",
    "        \"factorId\": \"5\",\n",
    "        \"factorName\": \"Factor 5\",\n",
    "        \"factorCategoryId\": \"IND\",\n",
    "        \"factorCategory\": \"Industry\",\n",
    "        \"factorReturn\": 0.2,\n",
    "    },\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Asset Data\n",
    "\n",
    "The asset data can be categorized into different components: \n",
    "   - **Factor Exposures**: factor loadings/zscores of assets in the universe.\n",
    "   - **Asset risk data**: Total risk, specific risk and specific return\n",
    "   - **Asset betas**: Historical beta and model-predicted beta\n",
    "   - **Market data**: Daily return, price, market capitalization, trading volumes, and dividend yields\n",
    "   \n",
    "   \n",
    "| Field                    | Optional? | Description                                                                                                                                                                                                   \n",
    "|--------------------------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
    "| universe                 | No        | The model universe uploaded as an array of identifiers. The identifier type must match the model's universe identifier  |\n",
    "| factorExposure           | No        | Array of dictionaries of factorId to the factor exposure of each asset in the universe.  There is no need to include a factorId in the map if the asset has zero exposure to the factor.|\n",
    "| specificRisk             | No        | Array of annualized specific risk. Represented in percent units (i.e. for 10%, use 10). The order in this array must match the order of the universe array. |\n",
    "| totalRisk                | No        | Array of annualized total risk. Represented in percent units (e.g., for 10%, use 10). The order in this array must match the order of the universe array. |\n",
    "| specificReturn           | No        | Array of specific returns. Assign `None` for missing or N/A values. The order of this array must match the order of the universe array.|\n",
    "| estimationUniverseWeight | No        | Array of weights in the estimation universe. The order of this array must match the order of the universe array. Assign `None` if an asset is not in the estimation universe.|\n",
    "| predictedBeta            | No        | Array of predicted betas. Assign `None` for missing or N/A values. The order of this array must match the order of the universe array.|\n",
    "| historicalBeta           | No        | Array of historical betas. Assign `None` for missing or N/A values. The order of this array must match the order of the universe array.|\n",
    "| globalPredictedBeta      | Yes       | Array of predicted betas relative to a global market. This measure is not applicable to all models. Assign `None` for missing or N/A values. The order of this array must match the order of the universe array. |\n",
    "| dailyReturn              | Yes        | Array of asset daily returns. Represented in percent units (e.g., for 10%, use 10). Assign `None` for missing or N/A values. The order in this array must match the order of the universe array. |\n",
    "| price                    | Yes        | Array of asset prices. Assign `None` for missing or N/A values. The order in this array must match the order of the universe array. |\n",
    "| currency                 | Yes        | Array of currencies. Assign `None` for missing or N/A values. The order in this array must match the order of the universe array.|\n",
    "| capitalization           | Yes        | Array of asset market capitalizations in trading currency. Assign `None` for missing or N/A values. The order in this array must match the order of the universe array. |\n",
    "| dividendYield            | Yes        | Array of dividend yields. Assign `None` for missing or N/A values. The ordering of this array must correspond to the ordering of the universe array |\n",
    "| tradingVolume            | Yes        | Array of asset daily trading volumes. Assign `None` for missing or N/A values. The ordering of this array must correspond to the ordering of the universe array |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "asset_data = {\n",
    "    \"universe\": [\"ASSET1\", \"ASSET2\", \"ASSET3\", \"ASSET4\", \"ASSET5\"],\n",
    "    \"factorExposure\": [\n",
    "        {\"1\": 0.48760401340992243, \"2\": -0.590034096365603},\n",
    "        {\"3\": -0.8017494714963063, \"4\": 0.7862451506530919},\n",
    "        {\n",
    "            \"5\": -0.253403252938764,\n",
    "        },\n",
    "        {\n",
    "            \"1\": 1.7,\n",
    "            \"2\": 0.08459557023760955,\n",
    "            \"3\": 1.8459557023760955,\n",
    "            \"4\": 0.8459557023760955,\n",
    "            \"5\": -1.8459557023760955,\n",
    "        },\n",
    "        {\"2\": -0.607987433173778},\n",
    "    ],\n",
    "    \"totalRisk\": [14.324714301179963, 17.071151010504597, 13.874122072602503, 14.479219370106968, 15.6372307010737],\n",
    "    \"specificRisk\": [6.267696111058748, 7.348880522500647, 7.866871501833286, 5.835214680084606, 11.76129276971361],\n",
    "    \"specificReturn\": [\n",
    "        0.5659048855060385,\n",
    "        0.37595374172320617,\n",
    "        -0.24473722135505294,\n",
    "        -0.9616000320458231,\n",
    "        -0.7699674832309915,\n",
    "    ],\n",
    "    \"predictedBeta\": [\n",
    "        0.8376437537112529,\n",
    "        0.8825284596124832,\n",
    "        0.8658162312132138,\n",
    "        1.1663832117848132,\n",
    "        1.1638549355761407,\n",
    "    ],\n",
    "    \"historicalBeta\": [\n",
    "        0.9699683237602925,\n",
    "        1.184349884914826,\n",
    "        1.1478458610614193,\n",
    "        1.1073962331683949,\n",
    "        0.8610028132979528,\n",
    "    ],\n",
    "    \"estimationUniverseWeight\": [None, 0.5, 0.2, 0.3, None],\n",
    "    \"dailyReturn\": [\n",
    "        0.36864638955606754,\n",
    "        -0.16247541720716152,\n",
    "        -0.14959565439528988,\n",
    "        0.3449230613583272,\n",
    "        0.4624063629113383,\n",
    "    ],\n",
    "    \"currency\": [\"USD\", \"USD\", \"USD\", \"USD\", \"USD\"],\n",
    "    \"price\": [146.85268872446377, 112.66584268302705, 102.8189727997599, 122.91632409053749, 131.95674622727046],\n",
    "    \"capitalization\": [2310528821.941124, 8391014828.775409, 7744577643.201652, 1873806225.329452, 6606600945.467762],\n",
    "    \"issuerMarketCap\": [\n",
    "        8694261819.837845,\n",
    "        7063154593.331768,\n",
    "        1761291245.7391672,\n",
    "        4251457616.7812405,\n",
    "        5264161805.272642,\n",
    "    ],\n",
    "    \"dividendYield\": [4.578537816528908, 3.7261774596135204, 3.4343027973124207, 3.413227950682553, 2.175273338412861],\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. Covariance Matrix\n",
    "\n",
    "The covariance matrix is represented as a 2D array, where each row corresponds to a factor, and the elements in the inner arrays represent the covariances (in daily variance units) between that factor and every other factor. The ordering of both the rows and columns aligns with the ordering of the factor data list in the payload. The first array below contains variance-covariance data for the factor with ID \"1\" and the first element is the variance of the factor itself. The second element is the covariance between the first and second factor, and so on. The covariance matrix must be a square matrix, meaning it has the same number of rows and columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "covariance_matrix = [\n",
    "    [0.9864, 0.026, 0.0965, 0.067, 0.0502],\n",
    "    [0.026, 0.6192, 0.0272, 0.0437, 0.0185],\n",
    "    [0.0965, 0.0272, 0.8771, 0.0863, 0.0407],\n",
    "    [0.067, 0.0437, 0.0863, 0.9589, 0.0333],\n",
    "    [0.0502, 0.0185, 0.0407, 0.0333, 0.5216],\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. Risk Free Rates \n",
    "\n",
    "The currency rates data is represented as an object containing information about currency exchange rates and risk-free rates. It includes the following components: `currency`, an array of strings representing the currency tickers; `exchangeRate`, an array of daily USD exchange rates, where the ordering corresponds to the ordering of the currencies; and `riskFreeRate`, an array of annualized monthly risk-free rates (in percentage), also ordered to match the currencies. If the model has a numeraire different from USD, the exchangeRate will be an array of daily exchange rates in that currency\n",
    "\n",
    "- currency: Array of currency tickers\n",
    "- exchangeRate: Array of daily USD (or the risk model numeraire if not USD) exchange rates, ordered to match the currencies\n",
    "- riskFreeRate: Array of annualized monthly risk-free rates (in percentage), ordered to match the currencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "currency_data = {\n",
    "    'currency': ['USD', 'EUR', 'GBP'],\n",
    "    'exchangeRate': [1.0, 0.85, 0.75],\n",
    "    'riskFreeRate': [0.01, 0.005, 0.007],\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Issuer Specific Covariance\n",
    "\n",
    "The issuer-specific covariance data is represented as an object that captures the covariance between two assets in daily variance units. It consists of three components: universeId1, an array of strings representing the first set of universe identifiers, which must exist in the asset universe; universeId2, an array of strings representing the second set of universe identifiers, also required to exist in the asset universe; and covariance, an array of numbers where each value represents the covariance between the corresponding identifiers at the same index in universeId1 and universeId2.\n",
    "\n",
    "- universeId1: Array of assets with issuer specific covariance to the asset in universeId2 at the same index. Each asset must also be present in the Asset Data universe\n",
    "- universeId1: Array of assets with issuer specific covariance to the asset in universeId1 at the same index. Each asset must also be present in the Asset Data universe\n",
    "- covariance: Array of the covariance between universeId1 and universeId2 at the same index. In daily variance units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "issuer_specific_covariance = {\n",
    "    'universeId1': ['ASSET1', 'ASSET2'],\n",
    "    'universeId2': ['ASSET3', 'ASSET5'],\n",
    "    'covariance': [0.03754, 0.01234],\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Factor Portfolios\n",
    "\n",
    "The factor portfolios data is represented as an object that defines a set of assets combining to form a portfolio representing a single factor. This portfolio has unit exposure to the specified factor and zero exposure to all other factors, making it useful for mapping intraday factor return data. It consists of two components: `universe`, an array of strings representing the universe of the factor portfolios, which must be a subset of the model asset universe; and `portfolio`, an array of dictionaries containing factor identifiers and their corresponding portfolio weights.\n",
    "\n",
    "The portfolios array contains objects with the following fields:\n",
    "- factorId: The ID of the factor corresponding to the factor data's factorIds\n",
    "- weights: An array of weights for each asset ID in the universe, corresponding to the ordering of the universe. The weights must sum to 1 and can include zero values (null values are not allowed)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_portfolios = {\n",
    "    \"universe\": [\"ASSET2\", \"ASSET3\", \"ASSET4\"],\n",
    "    \"portfolio\": [\n",
    "        {\"factorId\": \"1\", \"weights\": [0.3545899480088869, 0.2880743474246568, 0.35733570456645636]},\n",
    "        {\"factorId\": \"2\", \"weights\": [0.4390053364378467, 0.2907941789867976, 0.2702004845753557]},\n",
    "        {\"factorId\": \"3\", \"weights\": [0.3547375024699844, 0.29918415302858037, 0.3460783445014351]},\n",
    "        {\"factorId\": \"4\", \"weights\": [0.5821881487398893, 0.06066318669175055, 0.35714866456836014]},\n",
    "        {\"factorId\": \"5\", \"weights\": [0.4078609724431924, 0.39878265902389126, 0.1933563685329163]},\n",
    "    ],\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "### Upload Data\n",
    "\n",
    "Now we are ready to upload risk model data. Note you can only upload on dates that are in the model's calendar.\n",
    "The data must be uploaded in a specific format, which is defined in the `RiskModelData` class. The data must be uploaded for each date in the calendar, one date at a time.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = {\n",
    "    'date': '2021-01-13',\n",
    "    'factorData': factor_data,\n",
    "    'covarianceMatrix': covariance_matrix,\n",
    "    'assetData': asset_data,\n",
    "    'currencyRatesData': currency_data,\n",
    "    'issuerSpecificCovariance': issuer_specific_covariance,\n",
    "    'factorPortfolios': factor_portfolios,\n",
    "}\n",
    "\n",
    "risk_model = FactorRiskModel.get(model_id)\n",
    "risk_model.upload_data(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Important\n",
    "\n",
    "For large payloads, use the `max_asset_batch_size` parameter to split the data into smaller batches. This is particularly important for large models (asset universe larger than 10,000), as it helps prevent timeouts during the upload process. Typically, a batch_size of `10000` works well. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "risk_model.upload_data(data, max_asset_batch_size=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Check which days have data posted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `get_dates` method returns a list of dates for which data has been posted to the risk model. This is useful for verifying whether the data upload was successful. Upon completing the historical data backfill, all dates returned by this method should match the dates in the model's calendar. If any dates are missing, it may indicate that the data upload was incomplete or has not occurred yet.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dates_posted = risk_model.get_dates()\n",
    "print(dates_posted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Uploading Partial data\n",
    "The `upload_data` method can be used to upload partial data, which is useful for updating subsets of data without re-uploading the entire dataset. Data that can be uploaded partially includes `factorData` and `covarianceMatrix`, `assetData`, `factorPortfolios`, and `issuerSpecificCovariance`. Note that `factorData` and `covarianceMatrix` must already be uploaded when uploading partial data. If not, you must upload them first. The partial data must conform to the same structure as the full data upload but can represent a subset of the complete dataset.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uploading factor portfolios separately\n",
    "factor_portfolios = {'date': '2021-01-13', 'factorPortfolios': factor_portfolios}\n",
    "\n",
    "risk_model.upload_data(factor_portfolios, max_asset_batch_size=1000)\n",
    "\n",
    "# Uploading issuer specific covariance separately\n",
    "issuer_specific_covariance = {'date': '2021-01-13', 'issuerSpecificCovariance': issuer_specific_covariance}\n",
    "risk_model.upload_data(issuer_specific_covariance, max_asset_batch_size=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7: Enhance Factor Descriptions and Tool Tips\n",
    "\n",
    "The last step is adding tooltips and descriptions to the risk model factors. We highly encourage you to do this for every non-binary factor in your model (such as style factors) so that Marquee UI users of your model can leverage the tooltips and descriptions to better understand how the factors were constructed and what they represent. The snippet below should be repeated for each factor for which you are updating descriptions and tooltips. The `identifier` field must match the `factorId` in the factor data you uploaded earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "identifier = '3'\n",
    "tooltip = 'Short description that appears when you hover over the factor name on our UI.'\n",
    "description = 'Longer description that appears on the portfolio drill-down page of this factor.'\n",
    "glossary_description = 'Longest description to describe the factor in depth on our risk model glossary page.'\n",
    "\n",
    "factor = RiskModelFactor(\n",
    "    identifier=identifier,\n",
    "    type_=FactorType.Factor,\n",
    "    tooltip=tooltip,\n",
    "    description=description,\n",
    "    glossary_description=glossary_description,\n",
    ")\n",
    "\n",
    "risk_model.save_factor_metadata(factor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
